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What  does  SCRAM  mean? 


■  Go  away! 

■  Secure  Continuous 
Remote  Alcohol 
Monitoring 

□  As  modeled  here  by 
Lindsay  Lohan 

■  Schedule  Compliance 
Risk  Assessment 
Methodology 
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SCRAM 


Schedule 

Compliance 

Risk 

Assessment 

Methodology 


Collaborative  effort: 

□  Australian  Department  of  Defence  - 
Defence  Materiel  Organisation 

□  Systems  and  Software  Quality 
Institute,  Brisbane,  Australia 

□  Software  Metrics  Inc.,  Haymarket,  VA 
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DMO  SCRAM  Usage 


■  SCRAM  has  been  sponsored  by  the  Australian  Defence 
Materiel  Organisation  (DMO) 

□  To  improve  our  Project  Schedule  Performance  in  response  to 
Government  concern  as  identified  by  the  Australian  National 
Audit  Office  (ANAO) 

■  ANAO  is  equivalent  to  the  US  Government  Accountability  Office 


(GAO) 


■  DMO  equips  and  sustains  the  Australian  Defence  Force 


(ADF) 


□  Manages  230+  Major  Capital  Equipment  Projects  &  100  Minor 
(<$20M)  defence  projects 
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DMO  SCRAM  Usage  (cont.) 

■  SCRAM  has  evolved  from  our  reviews  of  troubled 
programs 

□  Schedule  is  almost  always  the  primary  concern  of  program 
stakeholders  (delays  to  war  fighter  capability  unacceptable) 

□  SCRAM  is  a  key  component  of  our  initiative  to  identify  and 
remediate  (and  eliminate)  root  cause  of  schedule  slippage 
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Topics 

■  Three  Common  Questions  Addressed  by  SCRAM 

■  Benefits  of  Using  SCRAM 

■  SCRAM  Key  Principles 

■  SCRAM  Process 

■  Future  plans  for  SCRAM 
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Three  Common  Questions 

■  SCRAM  addresses  three  fundamental  questions. 


1.  Why  is  schedule  slipping? 

■  Root  cause  analysis 

2.  Is  the  schedule  credible? 

■  Assess  risk  and  identify  Issues  (including  estimated  rework) 

■  Assess  BoEs  (Basis  of  Estimate) 

■  Perform  schedule  “Health  Check” 

■  Perform  Monte  Carlo  analysis  using  inputs  from  other  SCRAM 
areas 

3.  How  can  future  slips  be  prevented? 

■  General  recommendations  based  on  SCRAM  review  findings 

■  Guidance  on  “leading  indicators”  of  slippage 


#SSQi 

Systems  &  Software  Quality  Institute 


Australian  Government 


£?'’  Department  of  Defence 

Defence  Materiel  Organisation 


7 

Software  Metrics  Inc. 


What  SCRAM  is  Not 


■  Not  an  assessment  of  technical  feasibility 

■  Not  an  assessment  of  process  capability 

□  However,  may  be  identified  and  treated  as  an  issue  if  process 
performance  is  identified  as  contributing  to  slippage 
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Why  is  schedule  slipping? 

■  Program  managers  are  flooded  with  a  wealth  of  data  and 
details 

□  Challenge  is  to  organize  all  of  this  information 

■  Identify  cause(s)  of  slippage 

■  Schedule  slippage  is  a  symptom  of  other  factors 

■  Take  effective  action  to  address  problems 

□  Organizing  the  information  based  on  SCRAM  should: 

■  De-clutter  the  massive  amounts  of  information  on  a  project 

■  Relate  the  different  issue  areas  to  each  other 

■  Highlight  missing  information 


■  SCRAM  is  based  on  a  “Root  Cause  Analysis  of 
Schedule  Slippage  -  RCASS”  model 


#SSQi 

Systems  &  Software  Quality  Institute 


Australian  Government 


&x**  Department  of  Defence 

Defence  Materiel  Organisation 


9 

Software  Metrics  Inc. 


Root  Cause  Analysis  of  Schedule  Slippage 
(RCASS)  Model 


■  After  many  assessments, 
refined  RCASS  for 
guidance  in: 

□  Categorizing  the  wealth  of 
data  and  details 

□  Assessing  the  causes  of 
slippage 

□  Recommending  a  going- 
forward  plan 


#SSQi 

Systems  &  Software  Quality  Institute 


Adapted  from  Integrated  Analysis  Model  in  McGarry  et  al., 

Practical  Software  Measurement:  Objective 
Information  for  Decision  Makers 


Australian  Government 


r?1’  Department  of  Defence 

Defence  Materiel  Organisation 


10 

Software  Metrics  Inc. 


SCRAM-RCASS 
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Root  Cause  Analysis  -  Examples 


[  Stakeholders  ] 


[Subcontractors]  [  Requirements  ] 


Functional 

Assets 


V 


\  f 


[Staffing  &  Effort]^ 


Management 

& 

Infrastructure 


Schedule  & 
Duration 


/ 

l]\ 

v  Schedule 
'  Execution 

■  Stakeholders 

□  “Our  stakeholders  are  like  a 
100-headed  hydra  -  everyone 
can  say  ‘no’  and  no  one  can  say 
‘yes’.” 


■  Requirements 

□  Misinterpretation  of  a  communication  standard  led  to  an 
additional  3,000  requirements  to  implement  the  standard. 
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Root  Cause  Analysis  -  Examples 


■  Subcontractor 

□  Subcontractor  omitting  processes  in  order  to  make  delivery 
deadlines  led  to  integration  problems  with  other  system 
components. 

■  Functional  Assets  (COTS/MOTS) 

□  Commercial-off-the-shelf  (COTS)  products  that  do  not  work  as 
advertised,  resulting  in  additional  work  or  replacement  with 
different  products. 

□  Underestimating  amount  of  software  code  that  must  be 
written/modified  in  a  legacy  system. 


[  Stakeholders  ] 


[Subcontractors]  [  Requirements  ) 


Functional 

Assets 


V 


[Staffing  &  Effort^ 


Management 

& 

Infrastructure 


Schedule  & 
Duration 


/ 
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v  Schedule 
'  Execution 
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Root  Cause  Analysis  -  Examples 

■  Workload 

□  Optimistic  estimates 

■  Source  lines  of  code  underestimated 

■  Contract  data  deliverables  workload  often  underestimated  by  both 
contractor  and  customer 


[  Stakeholders  ] 


[Subcontractors]  [  Requirements  ] 


Functional 

Assets 


V 


\  f 


[Staffing  &  Effort]^ 


Management 

& 

Infrastructure 


Schedule  & 
Duration 


/ 
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v  Schedule 
'  Execution 

■  Staffing  &  Effort 

□  High  turnover,  especially  among  experienced  staff 


■  Schedule  &  Duration 

□  Area  of  primary  interest 
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Root  Cause  Analysis  -  Examples 

■  Schedule  Execution 

□  Schedule  replans  are  not  communicated  to  program  staff  or 
stakeholders 

□  Lack  of,  or  poorly  integrated,  master  schedule 

□  Integrated  schedule  elements  not  statused  consistently  across 
program.  Actual  status  unknown. 

□  External  dependencies  not  integrated  or  tracked 

■  Rework 


[  Stakeholders  ] 


[Subcontractors]  [  Requirements  ] 


Functional 

Assets 


V 


[Staffing  &  Effort]^ 


Management 

& 

Infrastructure 


Schedule  & 
Duration 


/ 

l]\ 

v  Schedule 
'  Execution 

□  Often  underestimated  or  not  planned  for  (e.g.  defect  correction) 

■  Management  &  Infrastructure 

□  Lack  of  adequate  test  facilities  (in  terms  of  fidelity  or  capacity) 


#SSQi 

Systems  &  Software  Quality  Institute 


Australian  Government 


£?'’  Department  of  Defence 

Defence  Materiel  Organisation 


15 

Software  Metrics  Inc. 


Three  Common  Questions 


1.  Why  is  schedule  slipping? 

□  Root  Cause  Analysis  of  Schedule  Slippage  -  RCASS  model 
guides  the  analysis  approach 

2.  Is  the  current  schedule  credible? 

□  Assess  the  risks  and  issues 

□  Assess  the  BoEs  (Basis  of  Estimate) 

□  Perform  “Schedule  Health  Checks” 

□  Perform  Monte  Carlo  analysis 

3.  How  can  future  slips  be  prevented? 
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Assess  the  Risks  and  Issues 


■  Are  risks  and  issues  understood  and  managed? 

■  What  mitigations  are  in  place  to  address  the  risks? 

■  Have  the  issues  been  analyzed  to  determine  corrective 
actions? 

□  Are  corrective  actions  being  managed  through  to  closure? 

■  Is  there  contingency  in  the  schedule  if  risks  are  realized? 

□  Or  is  the  schedule  so  tight  that  nothing  can  go  wrong? 
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Assess  the  BoEs 


■  Technical  expertise  is  essential 

■  Basis  of  estimate  will  vary  by  phase  and  activity 

□  Requirements 

□  Source  Lines  of  Code 

□  Test  cases/procedures 

■  Evidence  of  use  of  historical  data,  models 
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Schedule  Health  Checks 

■  To  evaluate  schedule  construction  and  logic 

□  Includes  analyses  of  task  dependencies,  task  constraints,  and 
available  schedule  float 

■  WBS  and  Master  Schedule  are  reviewed  for  alignment 

■  Government,  Prime,  and  Subcontractor  schedule 
integration  /  alignment  is  reviewed 


■  Ensure  external  dependencies  are  included  and  linked  in 
the  schedule 


□  Interfaces,  resources,  facilities,  Government  Furnished 
Equipment  (GFE),  test  assets  etc. 
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Schedule  Health  Checks  (cont.) 


■  Allocate  three  point  estimates  to  tasks  on  critical  and 
near-critical  path  based  on  identified  risk  from  RCASS 

□  optimistic,  pessimistic  &  most  likely  task  duration 

■  Perform  Schedule  Risk  Simulation  (e.g.  Monte  Carlo) 
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Monte  Carlo  Analysis  Example 


Open  Ran  Professional  ■  [Risk  Histogram  View  £Tl|o*]iX  j 
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Three  Common  Questions 


1.  Why  is  schedule  slipping? 

□  Root  Cause  Analysis  of  Schedule  Slippage  -  RCASS  model 
guides  the  analysis  approach 

2.  Is  the  current  schedule  credible? 

□  Assess  the  BoEs  (Basis  of  Estimate) 

□  Perform  schedule  “health  checks” 

□  Perform  Monte  Carlo  analysis 

3.  How  can  future  slips  be  prevented? 

□  General  recommendations  based  on  SCRAM  assessment 

□  Guidance  on  measurements  to  serve  as  “leading  indicators”  of 
future  slippage 
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SCRAM  Recommendations  -  Examples 

■  Clarify  the  delivery  scope  (requirements  and  acceptance 
criteria) 

■  Create  an  Integrated  Master  Schedule 

■  Test  Procedure  development  should  be  more  closely 
tracked  and  time  should  be  added  to  the  schedule  for 
their  review  and  correction 

■  Additional  time  in  all  test  phases  should  be  added  for  re¬ 
running  tests  that  fail  or  are  blocked 

■  Enhance  fidelity  of  integration  lab  to  improve  defect 
identification 
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Root  Cause  Analysis  of  Schedule  Slippage 
Model 


■  Provides  guidance  for 
collection  of 
measurements 

□  For  visibility  and  tracking  in 
those  areas  where  there 
are  risks 
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Topics 

■  Three  Common  Questions  Addressed  by  SCRAM 

■  Benefits  of  Using  SCRAM 

■  SCRAM  Key  Principles 

■  SCRAM  Process  Reference  /  Assessment  Model 

■  Future  plans  for  SCRAM 
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SCRAM  Benefits 


■  SCRAM  root-cause  analysis  model  (RCASS)  useful  in 
communicating  the  status  of  programs  to  all  key 
stakeholders 

□  Particularly  executive  management 

■  Identifies  Root  Causes  of  schedule  slippage  and  permits 
early  remediation  action 

■  Provides  guidance  for  collection  of  measures 

□  Provides  visibility  and  tracking  for  those  areas  where  there  is  risk 


■  Provides  confidence  in  the  schedule 
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SCRAM  -  Benefit 


■  Validate  schedule  before  execution 


■  Widely  applicable 

□  SCRAM  can  be  applied  at  any  point  in  the  program  life  cycle 

□  SCRAM  can  be  applied  to  any  major  system  engineering  activity 
or  phase 


■  Examples 

□  Software-Hardware  Integration 

□  Aircraft  Flight  Testing 

□  Installation/integration  of  systems  on  ship 

□  Logistics  ERP  application  roll  out  readiness 
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Topics 

■  Three  Common  Questions  Addressed  by  SCRAM 

■  Benefits  of  Using  SCRAM 

■  SCRAM  Key  Principles 

■  SCRAM  Process  Reference  /  Assessment  Model 

■  Future  plans  for  SCRAM 
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SCRAM  Key  Principles 


■  Minimal  Disruption 

□  Information  is  collected  one  person  at  a  time 

□  Interviews  typically  last  an  hour 


■  Independent 

□  Review  team  members  are  organizationally  independent  of  the 
program  under  review 


■  Non-advocate 


□  All  significant  issues  and  concerns  are  considered  and  reported 
regardless  of  origin  or  source  (Customer  and/or  Contractor). 


□  Some  SCRAM  reviews  have  been  joint  contractor/customer 
team  -  facilitates  joint  commitment  to  resolve  outcomes 
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SCRAM  Key  Principles  (cont.) 


■  Non-attribution 

□  Information  obtained  is  not  attributed  to  any  individual 

□  Focus  is  on  identifying  and  mitigating  the  risk 

■  Corroboration  of  Evidence 

□  Significant  Findings  and  Observations  based  on  at  least  two 
independent  sources  of  corroboration 

■  Rapid  turn-around 

□  One  to  two  weeks  spent  on-site 

□  Executive  briefing  presented  at  end  of  second  week 
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Topics 


■  Three  Common  Questions  Addressed  by  SCRAM 

■  Benefits  of  Using  SCRAM 

■  SCRAM  Key  Principles 

■  SCRAM  Process 

■  Future  plans  for  SCRAM 
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SCRAM  Process 


1.0  Assessment 
Preparation 


2.0  Project 
Awareness 
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3.0  Project 
Risk  /  Issue 
Identification 


4.0  Project 
Schedule 
Validation 


7.0  Observation 
&  Reporting 
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SCRAM  Team  Composition 

■  Assessment  conducted  by  a  small  team  including: 


□ 


□ 


□ 
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Engineering  Assessors 

■  Validate  WBS,  engineering-related  basis  of  estimates  (BoEs),  work 
load  estimates,  technical  risk  assessment 

Scheduler  experienced  in  the  project  schedule  tool 

■  Validates  schedule  -  conducts  schedule  health  checks 

■  Performs  Monte  Carlo  risk  modelling 

Other  project  domain  specialists  as  needed 

■  E.g.  Aeronautical  Flight  Test  Engineers 
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SCRAM  Key  Steps 

■  SCRAM  Team  briefs  the  Project  on  the  principles, 
purpose  and  approach  of  the  SCRAM 


■  The  Project  provides  the  SCRAM  team  with  an  initial 
overview  of  the  current  status  and  project  issues 

■  Project  Issues  and  Risks  are  confirmed  by  the  SCRAM 
Team  through  interviews,  reviewing  documentation  and 
other  project  assets 


■  Schedule  health  checks  and  Monte  Carlo  analysis  are 
performed 
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SCRAM  Key  Steps  (cont.) 


■  Executive  out  brief  is  prepared  and  presented 

□  Observations,  findings  and  recommendations 

□  Presentation  structured  using  the  RCASS  model 

■  Shows  cause  and  effect  linkage 

□  Findings  allocated  a  risk  code  rating 

□  Presented  at  the  end  of  the  second  week 

■  The  final  report  is  prepared  and  delivered  (an  additional 
two  weeks) 
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SCRAM  Findings  -  Examples 

■  Sample  Findings  with  Risk  Code  Rating 

H  POSITIVE: 

■  Functional  requirements  based-lined  and  agreed;  no  evidence  was 
identified  of  requirements  churn  or  creep 


POTENTIAL  RISK: 

■  Limited  schedule  contingency  exists  for  further  rework 


■  HIGH  RISK: 

■  Lack  of  an  integrated  high-level  schedule  precludes  the  ability  to 
accurately  forecast  project  milestone  achievements 
□  13  major  schedules  not  integrated  at  the  program  level 
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Process  Reference  /  Assessment  Model 


■  Developed  as  an  ISO/IEC  15504  conformant  Process 
Reference  Model  and  Process  Assessment  Model 

□  Funded  by  the  Australian  Defence  Materiel  Organisation  (DMO) 

□  Developed  by 

■  Systems  and  Software  Quality  Institute  and  Software  Metrics  Inc. 

□  Delivered  June  2010 

□  The  models  are  publicly  available  to  download  from: 

http://www.scramsite.org 
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Topics 

■  Three  Common  Questions  Addressed  by  SCRAM 

■  Benefits  of  Using  SCRAM 

■  SCRAM  Key  Principles 

■  SCRAM  Process 

■  Future  plans  for  SCRAM 
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Future  Plans 


■  Currently  developed  Diagnostic  SCRAM  (D-SCRAM) 

□  Full  scale  application  of  the  method  to  evaluate  challenged 
projects  or  Projects  of  Concern. 

□  Used  to  assess  likelihood  of  schedule  compliance,  root  cause  of 
schedule  slippage  and  to  recommend  remediation  of  project 
issues 


■  Further  evolve  the  SCRAM  process  for: 


□  Pro-active  SCRAM  (P-SCRAM) 

■  To  be  conducted  prior  to  Contract  or  at  Integrated  Baseline  Review 
(IBR)  to  ensure  common  systemic  issues  are  avoided  before  the 
Program  Schedule  is  contracted  or  baselined 

□  Monitor  SCRAM  (M-SCRAM) 


Systems  &  Software  Quality  Institute 


Reduced  version  of  D-SCRAM  that  maybe  used  to  monitor  project 
status  -  project  health  check  performed  ad  hoc  or  conducted  to 
support  appropriate  Gate  Reviews 
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Future  Plans  (cont.) 


■  SCRAM  Training  &  Assessor  Qualifications 

■  SCRAM  Process  Reference  and  Assessment  Model 

□  Further  revisions 

■  Based  on  feedback  from  use  during  SCRAM  assessments  and 

■  Change  Requests  (Appendix  D  in  the  model) 


■  SCRAM  Assessment  Tool 

□  Prototype  has  been  used 

□  Underdevelopment 
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SCRAM 


QUESTIONS 


For  further  information  contact: 


#SSQi 

Systems  &  Software  Quality  Institute 


Govt  to  Govt  -  Adrian  Pitman:  adrian.pitman@defence.qov.au 
Australia  -  Angela  Tuffley:  a.tufflev@ssqi.orq.au 
USA  -  Betsy  Clark:  betsv@software-metrics.com 
USA  -  Brad  Clark:  brad@software-metrics.com 
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Acronyms 

■  ANAO  -  Australian  National  Audit  Office 

■  BoE  -  Basis  of  Estimate 

■  COTS/MOTS  -  Commercial  off  the  Shelf/Modified  off  the  Shelf 

■  DMO  -  Defence  Materiel  Organisation  (Australia) 

■  GAO  -  Government  Accounting  Office 

■  GFE  -  Government  Furnished  Equipment 

■  ISO/IEC  -  International  Organization  for  Standardization/International 
Electrotechnical  Commission 

■  ISO/IEC  1 5504  -  Information  Technology  -  Process  Assessment 

■  RCASS  -  Root  Cause  Analysis  of  Schedule  Slippage 

■  SCRAM  -  Schedule  Compliance  Risk  Assessment  Methodology 

■  SMI  -  Software  Metrics  Inc.  (United  States) 

■  SSQi  -  Systems  &  Software  Quality  Institute  (Australia) 
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